KNN Project Solution¶

Since KNN is such a simple algorithm, we will just use this "Project" as a simple exercise to test your understanding of the implementation of KNN. By now you should feel comfortable implementing a machine learning algorithm in R, as long as you know what library to use for it.

So for this project, just follow along with the bolded instructions. It should be very simple, so at the end you'll have an additional optional "bonus" project.

Get the Data¶

Iris Data Set¶

We'll use the famous iris data set for this project. It's a small data set with flower features that can be used to attempt to predict the species of an iris flower.

Use the ISLR libary to get the iris data set. Check the head of the iris Data Frame.

library(ISLR)

head(iris)

str(iris)

'data.frame':	150 obs. of  5 variables:
 $ Sepal.Length: num  5.1 4.9 4.7 4.6 5 5.4 4.6 5 4.4 4.9 ...
 $ Sepal.Width : num  3.5 3 3.2 3.1 3.6 3.9 3.4 3.4 2.9 3.1 ...
 $ Petal.Length: num  1.4 1.4 1.3 1.5 1.4 1.7 1.4 1.5 1.4 1.5 ...
 $ Petal.Width : num  0.2 0.2 0.2 0.2 0.2 0.4 0.3 0.2 0.2 0.1 ...
 $ Species     : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 1 1 1 1 1 ...

Standardize Data¶

In this case, the iris data set has all its features in the same order of magnitude, but its good practice (especially with KNN) to standardize features in your data. Lets go ahead and do this even though its not necessary for this data!

Use scale() to standardize the feature columns of the iris dataset. Set this standardized version of the data as a new variable.

stand.features <- scale(iris[1:4])

Check that the scaling worked by checking the variance of one of the new columns.

var(stand.features[,1])

Join the standardized data with the response/target/label column (the column with the species names.

final.data <- cbind(stand.features,iris[5])

head(final.data)

Train and Test Splits¶

Use the caTools library to split your standardized data into train and test sets. Use a 70/30 split.

set.seed(101)

library(caTools)

sample <- sample.split(final.data$Species, SplitRatio = .70)
train <- subset(final.data, sample == TRUE)
test <- subset(final.data, sample == FALSE)

Build a KNN model.¶

Call the class library.

library(class)

Use the knn function to predict Species of the test set. Use k=1

predicted.species <- knn(train[1:4],test[1:4],train$Species,k=1)

predicted.species

What was your misclassification rate?

mean(test$Species != predicted.species)

Choosing a K Value¶

Although our data is quite small for us to really get a feel for choosing a good K value, let's practice.

Create a plot of the error (misclassification) rate for k values ranging from 1 to 10.

predicted.species <- NULL
error.rate <- NULL

for(i in 1:10){
    set.seed(101)
    predicted.species <- knn(train[1:4],test[1:4],train$Species,k=i)
    error.rate[i] <- mean(test$Species != predicted.species)
}

library(ggplot2)
k.values <- 1:10
error.df <- data.frame(error.rate,k.values)

pl <- ggplot(error.df,aes(x=k.values,y=error.rate)) + geom_point()
pl + geom_line(lty="dotted",color='red')

You should have noticed that the error drops to its lowest for k values between 2-6. Then it begins to jump back up again, this is due to how small the data set it. At k=10 you begin to approach setting k=10% of the data, which is quite large.

Optional Assignment¶

You should feel pretty comfortable using KNN since its so simple. As an optional assignment, choose a data set from the UCI Machine Learning Repository and see if you can use the process laid out above to do your own classification!

Check out the data sets here

	Sepal.Length	Sepal.Width	Petal.Length	Petal.Width	Species
1	-0.8976739	1.015602	-1.335752	-1.311052	setosa
2	-1.1392	-0.1315388	-1.335752	-1.311052	setosa
3	-1.380727	0.3273175	-1.392399	-1.311052	setosa
4	-1.50149	0.09788935	-1.279104	-1.311052	setosa
5	-1.018437	1.24503	-1.335752	-1.311052	setosa
6	-0.535384	1.933315	-1.165809	-1.048667	setosa